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ABSTRACT 


This  paper  examines  a  longitudinal  model  of  a  manpower 
system  in  which  the  demands  for  effective  manpower  is  deter¬ 
mined  by  the  state  of  a  ?  _nite  Markov  chain,  and  there  are 
delays  in  training  an  adequate  supply  of  effective  manpower. 
We  present  an  operational  method  of  calculating  optimal 
appointment  policies.  This  calculation  can  in  turn  be 
used  to  find  the  equilibrium  operating  rules  for  the  system. 
The  model  is  a  useful  device  for  measuring  the  impact  of 
alternate  assumptions  about  continuation  rates,  manpower 
utilization  policies,  the  demands,  and  the  transition 
probabilities  in  the  demand  process. 


0.  INTRODUCTION 


This  paper  considers  the  problem  of  providing  an  adequate  supply  of  effec¬ 
tive  manpower  when  demand  for  manpower  is  uncertain  and  there  are  time  lags  in 
the  training  of  effective  personnel.  The  model  described  in  this  paper  is 
useful  In  three  specific  ways:  (i)  it  can  calculate  optimal  manpower  input 
policy  for  alternate  control  objectives,  (ii)  it  can  be  used  to  measure  the 
effects  of  alternate  training,  retention,  ai.i  utilization  policies  on  system 
performance,  and  finally  (iii)  it  can  measure  the  sensitivity  of  optimal  poli¬ 
cies  and  system  performance  to  various  assumptions  on  the  nature  of  the  sto¬ 
chastic  nature  of  the  demand  process. 

In  Section  J  we  present  a  model  for  the  flow  of  personnel.  This  model 
attempts  to  capture  the  essential  features  of  the  manpower  system  in  a  simple 
framework  by  utilizing  the  longitudinal  stability  of  various  manpower  cohorts. 
The  stochastic  nature  of  the  demand  process  is  described  in  Section  2.  At 
time  t  ,  the  demand  for  trained  manpower  is  determined  by  the  state  of  a 
finite  Markov  chain.  In  Section  3  we  present  two  alternate  control  objec¬ 
tives.  The  first  objective  is  to  minimize  the  expected,  weighted,  discounted 
square  of  the  error  between  the  effective  supply  and  demand  for  manpower.  The 
second  objective  is  to  minimize  the  expected  discounted  square  error  between 
the  actual  manpower  stocks  and  ideal  stocks  for  any  demand  state.  In  Section  4 
we  describe  the  form  of  the  optimal  policy  and  optimal  value  function.  Numeri¬ 
cal  solutions  are  presented  for  the  two  control  objectives.  In  Section  5  we 
outline  several  uses  for  the  model.  The  first  is  the  determination  of  the 
expected  manpower  stocks  when  the  optimal  appointment  policy  is  used.  Calcu¬ 
lation  of  the  equilibrium  stocks  can  then  be  used  to  calculate  the  expected 
cost  of  the  appointment  policy  along  with  some  expected  measure  o  system 
performance.  These  calculations  allow  us  to  measure  the  tradeoff  between 
cost  and  performance  for  alternate  manpower  policies.  In  Section  5  we  also 


consider  the  effect  of  errors  in  the  specification  of  the  stochastic  law  of 
motion.  We  find  it  is  relatively  easy  to  discover  the  effects  of  these  errors. 
Finally  in  Section  5  we  consider  the  effect  of  possible  alterations  in  the 
stock  of  manpower,  either  by  releasing  people  from  the  system  or  allowing 
personnel  inflow  that  has  a  past  accumulation  of  experience. 

The  appendix  is  devoted  to  the  theoretical  aspect  of  the  model  we  find 
that  the  optimization  procedure  is  a  hybrid  of  the  linear-qu  idratic  optimal 
control  problem  ( [ 1 1 , [A ] , [ 7  ] )  aud  the  Markov  decision  problem.  We  are  able 
to  show  that  the  optimal  decision  rules  are  linear  and  that  the  optimal  policy 
functions  are  quadratic.  In  addition,  under  special  assumptions  on  the  prob¬ 
lem  data  we  can  show  that  the  optimal  decision  rules  converge  to  an  equilibrium 
optimal  appointment  policy. 

The  model  was  motivated  by  the  problem  of  regulating  the  supply  of  naval 
aviators.  We  do  not  pretend  to  present  an  operational  solution  to  that  real 
problem,  however,  a  numerical  example,  using  the  naval  aviator  problem  as  a 
setting,  is  used  throughout  the  paper.  The  author  is  the  sole  source  of  the 
data  for  the  numerical  example. 

The  paper  is  based  on  a  longitudinal  manpower  flow  model;  Oliver  and 
Hookins  [5], [6],  that  has  been  used  in  a  similar  context  by  Grinold,  Marshall, 
and  Oliver,  [2],  and  Grinold  and  Oliver,  [3]. 
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1.  THE  FLOW  OF  MANPOWER 

This  section  describes  the  basic  manpower  flow  process.  For  a  more  detailed 
description  of  a  related  deterministic  model  see  [2]  and  [3]. 

The  state  of  the  manpower  system  is  observed  at  discrete  equally  spaced 
time  points  t  =  0,1,2, 3,  .  ..,  T  ;  where  T  is  a  planning  horizon.  The  time 
interval  (t  -  1 , t ]  after  time  t  -  1  and  up  to  and  including  time  t  will 
be  called  period  t  . 

At  each  time  t  individuals  in  the  system  are  classified  according  to 
their  length  of  service  (LOS) .  The  length  of  service  is  simply  the  number  of 
occasiotis  the  individual  has  been  classified.  Thus  individuals  entering  in 
period  t  will  have  length  of  service  equal  to  one  at  time  t  ,  since  they 
are  in  their  first  period  of  service  at  time  t  .  Individuals  entering  in 
period  t  -  1  that  are  still  present  at  timi  t  will  be  in  their  second 
period  of  service.  We  assume  that  m  is  the  maximum  number  of  periods  an 

4- 

individual  can  remain  in  the  system.  It  follows  that  individuals  in  the 

t  h 

system  at  time  t  in  their  m  period  of  service  must  leave  the  system  in 
period  t  +  1  . 

The  stock  of  manpower  at  time  t  is  described  by  the  length  of  service 
distribution  s(t)  ;  s  ( *" )  is  .  vector  with  m  components,  where  s^Ct)  is 
the  number  of  individuals  at  time  t  with  LOS  equal  to  k  . 

(1)  s(t)  =  [s  (t),s,(t),  ....  s  (t ) ] ’ 

I  l  m 

In  each  period  t  a  number  of  new  accessions  f(t)  enter  the  manpower 
system.  The  change  in  the  manpower  system  from  time  t  to  time  t  +-  1 
depends  on  three  factors:  the  stock  at  time  t  ,  the  accessions  in  period 
t  +  1  ,  and  the  continuation  rates  of  the  stock  of  manpower  at  time  t  .  The 
continuation  rate  q^  for  k  =  1,2,  ...,  m  -  1  is  defined  to  be  the  fraction 

Caution:  in  [2]  and  [3]  the  length  of  service  is  defined  to  be  the  number  of 
completed  periods  of  service,  and  is  then  one  less  than  the  measure  used  here. 


of  manpower  with  LOS  equal  to  k  ,  that  remains  in  the  system  an  additional 
period.  With  this  definition  we  have 


(2) 


k+1 


(t  +  1) 


j  f  (t  +  1)  if  k  =  0 
|qksk(t)  if  k  =  1,2, 


The  continuation  rates  are  intended  to  include  several  factors  that  in¬ 
fluence  the  flow  of  manpower:  the  organization's  retention,  promotion,  and 
retirement  policy,  natural  forces  such  as  mortality,  and  the  behavior  of 


personnel . 
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THE  DEMAND  AND  SUPPLY  OF  EFFECTIVE  MANPOWER 


This  section  describes  the  stochastic  demand  process  and  the  related 
deterministic  supply  process. 

The  demand  Z  for  effective  manpower  is  a  random  variable  that  can 

take  on  a  finite  number  of  values  z..,z„,  ...,  z  .  The  stochastic  law  of 

i  i  n 

motion  that  governs  the  evolution  of  the  process  z^  is  Markovian: 


Vzt+i '  zi  1  zt  "  V  ‘  rij 


We  assume  r  is  independent  of  t  ,  and  that  the  uncertain  demand  zt 
persists  through  period  t  +  1  . 

The  example  below  is  a  six  state  system  and  the  manpower  category  is 
naval  aviators.  State  1  of  the  Markov  chain  represents  a  calm  peacetime 
period,  while  State  2  represents  a  higher  condition  of  peacetime  preparation. 
States  3,  U,  5,  and  6  represent  various  stages  of  a  conflict.  Notice  the 
conflict  may  end,  by  a  return  to  State  1  or  2,  may  stay  at  the  same  stage  or 
it  may  proceed  to  a  higher  stage.  The  transition  probabilities  are  shown 
below  along  with  the  demand  for  effective  manpower  and  the  equilibrium  dis¬ 
tribution  of  the  Markov  chain. 

Let  i(t)  be  the  state  of  demand  at  time  t  :  i.e.  Z^_  =  .  The 

state  of  the  system  at  time  t  depends  on  two  factors:  the  stock  of  manpower 
s(t)  ,  and  the  state  of  demand,  i(t)  . 

We  shall  now  generalize  the  nation  of  continuation  rates  introduced  in 
(I; 2),  by  allowing  the  continuation  rates  to  depend  on  the  state  of  demand. 
Thus  we  have  continuation  rate  q^  and  the  law  of  motion  for  manpower  is 


.  (t  +  1)  .lf<t+i>  lf  k  =  0  I 

k+1  I  qJ,,.s1.(t)  if  k  -  1,2,  ....  m  -  l| 


Demand  State 


TABLE  1 


1 

2 

3 

4 

5 

6 

1 

0.90 

0.07 

0.03 

2 

0.5 

0.4 

0.1 

j^|  l 

3 

0.05 

0.5 

0.05 

0.4 

_ 1 

4 

0.1 

0.35 

0.2 

0.35 

5 

0.1 

0.6 

6 

0.1 

0.7 

Hi 

TABLE  2 


Length  of  Service 


1 

2 

3 

4 

5 

1 

0.9 

0.9 

C.  95 

0.95 

2 

0.9 

0.9 

0.95 

0.95 

0.95 

3 

0.95 

0.95 

0.9 

0.9 

0.8 

4 

0.95 

0.9 

0.9 

0.9 

0.9 

5 

0.95 

0.8 

0.9 

0.8 

0.8 

0.9 


6  0.95 


0.9 


0.9 


0.9 
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This  allows  for  alternate  retention,  mortality,  and  behavior  assumptions  for 
each  state  of  the  demand  process.  In  this  paper  we  shall  solve  numerical 
examples  with  n  =  6  states  and  m  =  8  maximum  years  of  service.  The  con¬ 
tinuation  rates  are  shown  in  Table  2. 

We  can  utilize  personnel  in  alternate  ways  depending  on  the  demand  en¬ 
vironment.  Let  d.,  be  the  contribution  to  effectiveness  of  each  individual 

lk 

in  the  system  with  length  of  service  equal  to  k  when  the  demand  environment 
is  i  .  The  total  effectiveness  is  then 


(4) 


m 


k=l 


diksk(t) 


and  the  excess  of  the  supply  of  effective  manpower  is  given  by 


ra 

(5>  J,  diksk(t)  -  zi  ■ 

k-1 

We  shall  allow  the  d^  to  be  nonnegative.  Consider  the  problem  of 
supplying  effective  naval  aviators.  Individuals  in  the  first  period  of  service, 
are  being  trained  to  fly;  thui.  they  are  not  making  any  contribution  to  meeting 
the  demand  for  naval  aviators.  In  addition,  it  takes  trained  and  qualified 
pilots  to  teach  new  recruits.  Suppose  one  teacher  is  needed  for  every  two 
pupils  in  state  i  .  Then  d^  =  -.5  .  Every  two  pilot  trainees  take  up  the 
services  of  one  instructor  who  is  therefore  not  eligible  to  meet  the  demand 
for  operational  pilots.  The  contributions  to  effectiveness  for  the  problems 


solved  in  this  paper  are  seen  in  Table  3. 


Demand 


i$Sp5»'P!Sp!S*¥S*5l!5!!^?pssss!S!^^ 


TABLE  3 
Utilization 
Length  of  Service 


1 

2 

3 

4 

5 

6 

7 

8 

1 

"0.5 

"0.5 

1 

1 

0.8 

0.8 

0.2 

0.2 

2 

"0.5 

“0.5 

1 

1 

1 

0.8 

0.3 

0.2 

3 

“0.5 

"0.25 

1 

1 

1 

1 

0.8 

0.0 

4 

"0.5 

"0.3 

1 

1 

1 

1 

0.8 

0.8 

5 

"0.5 

"0.3 

1 

1 

1 

1 

0.8 

0.8 

6 

"0.5 

~0. 3 

1 

1 

1 

1 

0.8 

0.8 
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3.  THE  CONTROL  OBJECTIVE 

This  section  formulates  two  quadratic  objective  functions  for  controlling 
the  flow  of  manpower  through  the  system.  We  selected  quadratic  objectives  for 
two  -masons.  First,  they  lead  to  a  model  for  which  results  can  le  readily 
calculated.  Second,  the  objectives  are  a  reasonable  measure  of  tne  departure 
of  actual  system  performance  from  an  ideal  performance. 

The  simplest  control  objective  is  to  minimize  the  expected  weighted 
squared  error  between  the  supply  and  demand  for  effective  manpower.  Let 
Wj  for  j  =  1,2,  ...,  n  be  positive  weights  that  sum  to  1.  If  i(t)  =  i  , 
and  s(t)  ,  then  the  expected  error  at  time  t  +  1  ,  as  a  function  of 
f  =  f  (t  +  1)  ,  is 

.  n  /  m-i 

(1)  E  [s(t),f(t  +  1)]  =  l  r  w.lz  -  f(t  +  1)  -  l  q 

j=l  J  3\J  k=l 

Our  second  control  objective  is  to  provide  a  smooth  flow  of  manpower 
through  the  system  and  to  come  as  close  as  possible  to  meeting  the  demand 
for  e.'jL’ctive  manpower.  If  the  manpower  system  was  always  in  state  j  , 

.  hen  an  ideal  distribution  would  satisfy 

m 

<2)  Vsk  ’  h  ■ 
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For  k  =  2,3,  m  define 


pjk  = qjk 


and  P  . ,  =1  . 

jl 

Combining  (4),  (3),  and  (2)  we  obtain 


jk  '  j  jk 


Jk  jk 


The  'iscrepancy  between  actual  and  ideal  is  measured  by 


l  (s,  (t  +  i)  -  s.,r 


Using  the  law  of  motion  (II; 3)  we  can  rewrite  (7)  as 


-I  4  n  J- 

(8)  E1J[s(t),f(t  +  1)]  =  (f(t  +  1)  -  Sjir  +  l  (qiksk(t)  -  sjk+1)‘ 

k-1 


This  is  a  measure  of  the  discrepancy  at  time  t  4-  1  conditioned  on  four 
facts;  manpower  s(t)  at  time  t  ,  demand  i  at  time  t  ,  accessions  f(t  +  1) 
in  period  t.  +  1  ,  and  demand  j  at  time  t  +  1  .  The  function  is  quadratic, 
strictly  convex  in  f(t  +  1)  and  convex  in  s(t)  .  The  expected  discrepancy 


4  **  J  j 

F.  [s(t),f(t  +  1)]  =  l  r  .  [s(t) ,  f  (t  +  1)] 

j=l  1J 


is  also  quadratic  convex  in  f  (strictly)  and  s  . 

We  note  that  it  is  possible  to  weigh  the  functions  EW[s(t),f(t  +  1)] 
in  order  to  place  greater  or  less  importance  on  the  expected  discrepancy  from 
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state  j  . 

For  either  objective  in  a  T  stage  problem,  where  s(t)  ,  and  f(t)  are 
the  observed  values  of  the  manpower  stocks  and  accessions  for  t  =  1,2,  ...,  T  , 
and  i(t)  Is  the  state  of  demand,  the  total  discounted  error  is 


(10) 


l  6t_1Ei(t)[s(t),f(t)] 
t=l 


where  0  <  6  <  1  is  a  discount  factor.  The  objective  is  to  minimize  the 
expected  value  of  (10)  over  all  possible  appointment  sequences. 
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4.  THE  OPTIMAL  APPOINTMENT  POLICY 

This  section  presents  two  examples  of  optimal  appointment  policies:  one 
for  each  of  the  objectives  presented  in  Section  3. 

A  theoretical  description  of  the  optimization  procedure  is  presented  in 
the  appendix.  We  shall  use  three  important  results  from  the  appendix: 


(i)  For  any  finite  planning  horizon  T  ,  and  any  demand  state  i  , 

there  is  an  optimal  appointment  policy  that  is  a  linear  function 
of  the  manpower  stock  s  .  For  our  model 


f  = 


m 


l 

k=l 


(ii)  For  any  finite  planning  horizon  T  ,  and  any  demand  state  i  ,  the 
minimum  expected  error  is  a  convex  quadratic  function  of  the  man¬ 
power  stocks  s  : 


Ini,T  ,  i,T  ,  i,T 
s  P  s  +  q  s  +  r 


(iii) 


As  the  planning  horizon  T  becomes  large,  the  values  of 

pi,T  ni,T  i.T  „  ,i  pi  i  i 

P  »  9  .  ,  r  ,  converge  to  b  ,  P  ,  q  ,  r 


In  the  examples  we  have  solved  to  date,  the  convergence  has  been  quite 
rapid.  For  the  first  example  below  a  1%  change  was  observed  between  a  20  and 
21  period  horizon.  In  the  second  example  convergence  was  nearly  perfect  in  5 
periods . 

The  first  solution  below  uses  the  objective  (3:1)  with  discount  factor 


( 1/18  if  j 

r  rr  ' 

j  ( 4/18  if  j 


1,2 

3, 4, 5, 6 


I 

C 


4  =  0.95  and  weights 
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T.ie  optimal  appointment  policy  is  shown  m  Table  1. 

For  the  second  control  objective,  (3:9),  there  is  a  considerable 

m 

simplification  in  the  optimal  decision  rule:  f  =  b„  +  7  b1k  .  In  the 

0  k-i  Bk 

second  case  b  =  0  for  k  >  1  .  Thus  f  =  b^  independently  of  the  current 
manpower  stock  s  . 

The  optimal  input  as  a  function  of  i  is  shown  in  Table  2.  In  this 
calculation  we  si  t  o  =0.95  and  did  not  weigh  the  various  terms  in  the  sum 
(3:9). 


.30 

0.0 

.10 

0 

.31 

.03 

.16 

0 

.35 

.29 

.39 

0 

.47 

.40 

.46 

0 

.36 

.28 

.32 

0 

.36 

.22 

.25 

0 
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5.  COST  PERFORMANCE  TRADEOFFS 


This  section  examines  the  use  of  the  manpower  planning  model  presented 
in  earlier  sections  to  measure  the  tradeoff  between  cost  and  system  performance. 
We  shall  consider  this  tradeoff  in  the  specific  contex;  of  the  naval  aviation 
problem.  In  particular,  we  shall  assume  that  costs  are  determine!  by  the  dis¬ 
tribution,  s(t)  ,  of  manpower  stocks. 

When  an  optimal  linear  appointment  policy  has  been  calculated,  then  it  is 
possible  to  express  the  evolution  of  the  manpower  stocks  in  a  linear  fashion 


bi°  *  ,1  buV°  If  k’° 


Vi(t  +  u  ■ 


if  k  =  1,2,  . . . ,  m  -  1 


This  can  be  written  in  matrix  rorm 


;(t  +  1)  =  Ais(t)  +  h1 


The  equilibrium  expected  values  of  s  as  a  function  of  the  demand  state 
j  can  be  calculated. 

Let  be  the  probability  that  the  previous  demand  state  was  i  given 

that  the  current  demand  state  is  j  .  These  are  the  inverse  transition  proba¬ 
bilities  for  the  Markov  chain.  When  the  Markov  chain  is  in  equilibrium,  a 
simple  application  of  Bayes's  Law  gives 


ji 


where  t  =  (tt  ,  .  .  .  ,  rr  )  is  the  stationary  distribution  vector  of  the  Markov 
1  n 

chain. 

Now  let  for  j  =  1,2,  ...,  n  be  the  expected  manpower  stock  in  demand 

state  i  when  an  optimal  linear  appointment  policy  is  used. 


Tne.  sJ  must  satisfy 
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l  ^,,(A1si+h1) 


for  j  =  1,2, 


This  system  of  n  *  m  equations  with  n  x  m  unknowns  can  be  solved  for 
the  equilibrium  expected  values  of  s^  .  For  the  first  objective  (3:1),  and 
the  appointment  policy  in  Table  1  of  Section  4,  we  obtain  the  equilibrium  ex¬ 
pected  values  following  in  Table  1. 

For  the  second  objective  (3:9)  the  expected  equilibrium  stock  levels  are 
shown  in  Table  2. 

With  these  figure^,  suitable  cost  data  and  the  vector  tt  ,  it  is  easy  to 
calculate  the  expected  cost  ner  period  of  operating  this  manpower  system  using 
the  optimal  linear  appointment  policy.  Let  c^  be  the  cost  of  an  individual 
in  the  k^  period  of  service.  The  equilibrium  expected  cost  per  period  is  then 


u  m  . 

I  ",  l  c  ,  s* 
.u.  i  ,  ,  ik  k 
i=l  k=J 


We  can  also  construct  a  measure  of  system  performance.  Suppose  we  choose 
the  expected  weighted  squared  discrepancy  between  the  demand  and  the  steady 
state  expected  supply  level.  This  would  be 


n  /  m  ,\2 

ili  •  Jx 


With  a  measure  of  expected  system  performance  and  a  measure  of  expected 
cost,  we  compare  alternate  manpower  policies.  For  example,  it  is  possible  to 
change  the  demands  z ^  and  recalculate  a  new  optimal  appointment  strategy. 

We  can  also  change  the  continuation  rates  q  ^  ,  or  the  utilization  policy 
cl  ;  note  these  changes  would  probably  produce  a  corresponding  change  in  the 
costs  c  ,  .  It  is  also  possible  to  change  the  objective  function  by  either 


TABLE  1 


371 

332 

298 

283 

268 

— 

254 

29 

30 

350 

325 

296 

279 

262 

243 

73 

57 

362 

328 

297 

282 

267 

251 

42 

41 

285 

329 

308 

270 

251 

216 

199 

66 

376 

279 

289 

276 

240 

223 

195 

179 

328 

348 

241 

252 

222 

194 

195 

175 

307 

279 

252 

241 

229 

217 

25 

26 

330 

295 

262 

243 

226 

209 

6? 

49 

318 

285 

257 

243 

229 

215 

36 

35 

371 

312 

273 

234 

217 

186 

m 

57 

373 

353 

281 

246 

209 

192 

168 

153 

357 

351 

289 

254 

203 

171 

169 

151 

using  alternate  weights  or  an  entirely  different  objective  function.  Using 
our  model.,  and  trying  alternate  manpower  policies  we  can  discover,  and  explore 
the  cost-rerfonnance  tradeoffs  that  exist. 

There  is  a  second  type  of  sensitivity  analysis.  If  the  probabilities  in 
the  Markov  matrix  are  changed,  this  amounts  to  an  alternate  assumption  about 
the  stochastic  nature  of  the  demand  process.  By  recall  culating  for  several  dif¬ 
ferent  values  of  the  transition  probabilities  we  can  measure  the  impact  of 
alternate  assumptions  concerning  the  demand  process.  A  related  question  is 
to  measure  the  impact  of  an  incorrect  assumption  about  the  demand  process. 

This  leads  to  an  easier  calculation.  Suppose  we  assume  the  transition  proba- 
bilii  ies  are  r  ,  when  they  actually  are  r  with  equilibrium  distribution 

vector  tt  .  In  this  case  the  appointment  policy  will  not  change.  However,  the 

calculations  ( 3 )  —  (6 )  would  have  to  use  the  values  r,.  and  it,  .  In  this  way 

i]  i 

we  can  measure  the  effect  of  incorrectly  specifying  the  probabilities. 

In  some  cases  it  is  possible  to  alter  the  manpower  stock  in  an  ad-hoc 
fashion.  In  the  language  of  the  na^al  aviator  problem,  we  can  either  bring  in 
reserves  or  allow  people  in  the  system  to  leave  before  their  obligated  service 
is  completed.  If  possibilities  of  this  sort  exist,  then  some  of  the  information 
we  have  calculated  can  help  in  designing  an  alteration  of  the  manpower  stock. 
Recall  that  the  minimum  expected  error  starting  in  demand  state  i  with  stock 
s  is 


Di  .  i  .  i 
sP  s  +  q  s  +  r 


The  increase  in  this  optimal  expected  error  as  a  function  of  an  increase  in  s 
is  simply 


2  sjpjk  +  %  • 


'  •  u  w-V-.. 


19 


This  tells  how  an  increase  or  decrease  in  s^  will  effect  the  minimum  expected 
error. 

In  addition,  if  the  current  demand  state  is  i  ,  then  the  current  manpower 
stock  s  should  be  contrasted  with  its  equilibrium  value  s*  .  It  is  in  the 
interest  of  system  stability  to  change  s  closer  to  its  equilibrium  value. 

It  is  even  possible  to  calculate  an  optimal  value  for  s  when  the  demand 
state  is  i  .  The  optimum  is  simply  -l/2(Pi)  ^(q*)'  ,  the  s  that  minimizes 
(7).  The  optimal  values  are  not  always  realistic,  as  we  indicate  below.  How¬ 
ever,  they  do  indicate  a  direction  in  which  favorable  changes  can  be  made. 

For  the  objective  (3:1)  we  find  the  following  optimal  distributions  shown 
in  Table  3.  Note  any  value  of  s^g  is  optimal. 

The  distributions  for  demand  states  i  =  1,2  are  unreasonable,  probably 
close  to  the  low  weight  attached  to  the  objective  for  those  states.  The  dis¬ 
tribution  for  States  3  and  4  is  quite  good.  It  indicates  that  a  large  increase 
in  the  three  year  group  would  have  a  favorable  impact,  as  well  as  a  large  in¬ 
crease  in  the  six  year  group. 

The  optimal  distributions  for  the  second  objective  (3:9)  are  more  reasonable. 

Those  values  shown  in  Table  4  should  be  contrasted  with  those  of  the  equil¬ 


ibrium  distribution  in  Table  2. 


288 

32 

474 

1  U 

“61 

481 

302 

159 

514 

363 

254 

415 

2 

■ 

244 

583 

184 

169 

682 

303 

272 

334 

303 

360 

332 

379 

355 

350 

377 

342 

330 

BLE  4 


296 

27 

256 

54 

143 

131 

130 

143 

91 

98 

80 

83 
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APPENDIX 

DISCRETE  TIME,  LINEAR-QUADRATIC ,  DYNAMIC  PROGRAMS  WITH  MARKOVIAN  PARAMETERS 


This  appendix  presents,  in  a  more  general  form,  the  theoretical  results 
used  in  the  paper.  We  employ  a  notation  close  to  that  of  the  text. 

The  state  of  the  system  with  T  +  1  periods  remaining  is  (s,i)  .  Given 
action  f  ,  the  state  of  the  systeir  with  \  periods  remaining  will  be 
(D*(s,f),j)  with  probability  r  ,  where,  for  each  i  ,  D'  (s,f)  is  a 
linear  function  of  s  and  f  . 

The  error  associated  with  choosing  f  in  state  (s.i)  is  E^(s,f)  , 
where  for  each  i  EX(s,f)  is  quadratic  and  convex  in  s  and  strictly  convex 
in  f  .  Future  errors  are  discounted  at  rate  . 

For  i  =  1,2,  ...,  n  define  V^Cs.i)  to  be  the  minimum  expected  discounted 
error  for  a  T  period  problem  that  starts  with  manpower  stocks  s  ,  and  demand 
state  i  . 

For  any  choice  of  f  ,  we  have 


vx+l(Sji)  -  £l(s’f)  +  6  l  rii7T(Di(3,f),j) 

j=l  J 


By  the  usual  principal  of  optimality  arguments  we  can  establish  that  the  func¬ 
tions  VT+^(s,i)  must  satisfy  the  following  functional  equation. 


vT,i  (5, f)  =  Min  Ei(s,f)  +  <5  l  r  ,  ,  V  (D1  (s,  f) ,  j ) 


where  V  (s,i)  =  0  for  each  s  and  i  . 
o 

Now  assume,  that  V^Cs.i)  is  a  convex  quadratic  function  of  s  for  each 
i  .  Thus  we  can  write  V^(s,i)  =  s'P^’^s  +  q^’^s  +  r^’^  where  PX ’ ^  is 
positive  semi-definite,  and  symmetric.  It  is  easy  to  show  that  V^, (D1 (s , f ) , j ) 
is  a  convex  quadratic  function  of  (s,f)  ,  and  therefore  that 


.  *  Mi  u.  V, 


.  .W'iR 


Ei(s,f)  +  6  l  r  V  (Di(s,f),j) 
1=1  lj 


is  ouadratic,  convex  in  s  ,  and  strictly  convex  in  f  • 


When  the  quadratic  function  (3)  is  differentiated  with  respect  to  f  , 


then  a  linear  equation  results,  and  the  unique  f  that  minimizes  (3)  as  a 


function  of  s  is  in  the  form 


i . i+l . 


fT+1 (s,i)  =  B  (s) 


i  T+l 

where  for  each  i  and  T  +  1  ,  B  ’  (s)  is  linear  in  s  .  When  the  optimal 


policy  (4)  is  substituted  into  (3)  we  obtain  a  convex  quadratic  expression  for 


VT+i<s,l>  • 


Thus  for  any  finite  horizon  T  the  optimal  value  and  policy  functions 


are,  respectively,  linear  and  quadratic.  We  now  turn  to  the  question  of  con¬ 


vergence  of  policies  as  the  planning  horizon  increases. 


Let  V  (s,i)  =  s’Pi,Ts  +  qi>Ts  +  ri,T 


be  the  optimal  value  function  starting 


in  state  i  for  an  5.  period  problem. 


When  the  discount  factor  6  is  strictly  less  than  1,  then  the  sequence 


V  (s,i)  will  remain  bounded  if  there  exists  a  policy  f(s,i)  such  that  this 


policy  leads  to  a  bounded  sequence  s(t)  for  any  initial  s(o)  and  any  evolu¬ 


tion  of  the  demand  sequence  i(t)  .  In  the  model  described  in  this  paper,  the 


policy  f(s,i)  =  0  will  insure  s(t)  =  0  for  t  >  m  .  The  general  question 


T  .  T 

of  convergence  of  V  (s,i)  and  B  (s,i)  is  difficult,  and  is  currently  the 


object  of  further  investigation  by  the  author. 


